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Abstract 



This paper is concerned with inference based on the mean function of a functional 
time series, which is dehned as a collection of curves obtained by splitting a continu- 
ous time record, e.g. into daily or annual curves. We develop a normal approximation 
for the functional sample mean, and then focus on the estimation of the asymptotic 
variance kernel. Using these results, we develop and asymptotically justify a testing 
procedure for the equality of means in two functional samples exhibiting temporal 
dependence. Evaluated by means of a simulations study and application to real data 
sets, this two sample procedure enjoys good size and power in finite samples. We 
provide the details of its numerical implementation. 
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1 Introduction 



Functional time series form a class of data structures which occurs in many applications, 
but several important aspects of estimation and testing for such data have not received 
as much attention as for functional data derived from randomized experiments. In the 
latter case, the curves can often be assumed to form a simple random sample, in partic- 
ular, the functional observations are independent. For curves obtained from splitting a 
continuous (in principle) time records into, say, daily or annual curves, the assumption 
of independence is often violated. This paper focuses on the methodology and theory for 
the estimation of the mean function of a functional time series, and on inference for the 
mean of two functional time series. Despite their central importance, these issues have 
not yet been studied. The contribution of this paper is thus two-fold: 1) we develop a 
methodology and an asymptotic theory for the estimation of the variance of the sam- 
ple mean of temporally dependent curves under model-free assumptions; 2) we propose 
procedures for testing equality of two mean functions in functional samples exhibiting 
temporal dependence. 

A functional time series {Xk, G Z} is a sequence of curves Xk{t), t G [a, 6]. After 
normalizing to the unit interval, the curves are typically defined as Xk{t) = X{k + t), < 
t < 1, where {X{u),u G M} is a continuous time record, which is often observed at 
equispaced dense discrete points. An example is given in Figure which shows seven 



consecutive functional observations. More examples are studied in Horvath and Kokoszka 



(2011) A central issue in the analysis of such data is to take into account the temporal 



dependence of the observations. The monograph of Bosq (2000) studies the theory of 
linear functional time series, focusing on the functional autoregressive model. For many 
functional time series it is however not clear what specific model they follow, and for 
many statistical procedures it is not necessary to assume a specific model. In this paper, 
we assume that the functional time series is stationary, but we do not impose any specific 
model on it. We assume that the curves are dependent in a very broad sense, which is 
made precise in Section 1.1 The dependence condition we use is however satisfied by 
all models for functional time series used to date, including the linear, multiplicative, 
bilinear and ARCH type processes. We refer to Hermann and Kokoszka (2010) and ? for 
examples. 

A direct motivation for the research presented in this paper comes from a two sample 
problem in which we wish to tests if the mean functions of two functional time series are 
equal. A specific problem, studied in greater depth in Section |4| is to test if the mean 
curves of certain financial assets are equal over certain periods. This in turn allows us to 
conclude whether the expectations of future market conditions are the same or different 
at specific time periods. In general, if the same mean is assumed for the whole time series, 
whereas, in fact, it is different for disjoint segments, the inference or exploratory analysis 
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Figure 1.1. The horizontal component of the magnetic field measured in one minute resolution 
at Honolulu magnetic observatory from 1/1/2001 00:00 UT to 1/7/2001 24:00 UT. 
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that follows will be faulty, as all prediction and model fitting procedures for functional 
time series start with subtracting the sample mean, viewed as an estimate of the unique 
population mean function. The same holds true for independent curves; if two subsamples 
have different mean functions, subtracting the sample mean function based on the whole 
data set will lead to spurious results. Despite the importance of two sample problems 



for functional data, they have received little attention. Recent papers of Horvath et al. 



(2009) and Panaretos et al. (2010) are the only contributions to a two sample problem 



in a functional setting which develop inferential methodology. Horvath et al. (2009) 



compare linear operators in two functional regression models. Panaretos et al. (2010) 
focus on testing the equality of the covariance operators in two samples of iid Gaussian 
functional observations; our paper focuses on the means of dependent (and non-Gaussian) 
observations. We develop the required methodology, justify it by asymptotic arguments, 
and describe its practical implementation. 

Any inference involving mean functions requires estimates of the variability of the sample 
mean. In iid functional samples, the sample covariance operator is used, but for func- 
tional time series this problem is much more difficult. For scalar and vector-valued time 
series, the variance of the sample mean is asymptotically approximated by the long-run 
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variance whose estimation has been one of the central problems of time series analysis, 
studied in textbooks, see e.g. Anderson (1971), [Brockwell and Davis (1991) [Hamilton 



(1994) , and dozens of influential papers, see ?, [Andrews (1991) and Andrews and Mon- 



ahan (1992) , to name just a few. Convergence of various estimators of the long-run 



variance has been established under several types of assumptions, including broad model 
specifications (e.g. linear processes), cumulant conditions, and various mixing conditions. 



Hermann and Kokoszka (2010) and Gabrys et al. (2010) advocated using the notion of 



L^-m-approximability for functional time series, as this condition is intuitively appealing 
and is easy to verify for functional time series models. We therefore develop a general 
framework for the estimation of the long-run covariance kernel in this setting. The long- 
run covariance kernel (equivalently operator) is an infinite dimensional object, precise 
definitions are given in Section [2| whose estimation has not been studied yet. [Hormann 



and Kokoszka (2010) studied the estimation of a long-run covariance matrix obtained by 



projecting this operator onto a finite dimensional basis. Their approach is based on the 
results for vector-valued time series, and they use cumulant-like assumptions which are 
difficult to verify for nonlinear functional time series. 

The long-run covariance kernel corresponds to the asymptotic variance in a normal ap- 
proximation for the sample mean of a scalar time series, but no central limit theorem 
for a general functional time series has been established yet (results for linear processes 
are established in Bosq (2000)). We provide such a generally applicable result as well 
(Theorem 2.1 ). 

The remainder of the paper is organized as follows. We conclude the introduction by 
defining the notion of dependence for functional time series which we use throughout 
this paper. Then, in Section [2| we state the asymptotic results for the mean of a single 
functional time series, with proofs developed in Section [5j Section |3] focuses on the 
problem of testing the equality of means of two functional samples which exhibit temporal 
dependence. In Section [4| we evaluate the finite sample performance of the procedures 
proposed in Section [3] by means of a simulation study and application to real data. The 
proofs of the theorems stated in Section [3] are collected in Section [6| 



1.1 Approximable functional time series 

We consider a stationary functional time series {^^(t), i G Z, t G [0, 1]}, which we can 
view as an error sequence in a more complex functional model, for example a regression 
model, as in [Gabrys et al. (2010)[ We assume that these errors are nonlinear moving 



averages Ei = f{Si, Si-i, . . .), for some measurable function / : S°° — > L^, and iid elements 
6i of a measurable space S. In all models used in practice S = L'^. To motivate the 
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construction below, it is useful to write the Ei as 



(1.1) 



1 ^i—m+li ^i—mi ^i—m—l 



•)• 



Under (1.1), the sequence {Si} is stationary and ergodic. The function / must decay suf- 



ficiently fast to ensure that the sequence {si} is weakly dependent. The weak dependence 
condition is stated in terms of an approximation by m-dependent sequences, namely, we 
require that 



(1.2) 

where 
(1,3) 



E 

m>l 



E / {e,{t) - ei,^{t)f dt 



.1/2 



< OO, 



f{Si, . . . 



r(m) r(m) 



-I,---), 



with the sequences {5^™'*} being independent copies of the sequence Note that the 



sum in (1.2) does not depend on i. 



The idea behind the above construction is that the function / decays so fast that the 
effect of the innovations 5i far back in the past becomes negligible; they can be replaced by 
different, fully independent innovations. If the Si follow a linear model Si = X]j>o '^ji^i-j)' 
condition ( |1.2 ) intuitively means that the approximations by the finite moving averages 
£i,m = Ylo<j<m'^ji^i-j) bccomc increasingly precise. This means that the operators Cj 



must decay sufficiently fast in an appropriate operator norm. We refer to |Hormann and 
Kokoszka (2010)] and ? for the details and examples of nonlinear functional time series 



satisfying (1.2) 



We also note that the general idea of using nonlinear moving averages (Bernoulli shifts) 
and imposing moment conditions to quantify dependence has been recently used in other 



contexts, see Wu (2005, 2007). The connections between such notions and the traditional 



mixing conditions, or other notions of weak dependence, e.g. that introduced by Doukhan 



and Louhichi (1999) , are only partially understood at present. In particular, it is not 
clear which functional time series models satisfy dependence conditions other than the 



approximability (1.2). 



2 Normal approximation and long— run variance for func- 
tional time series 

In this section, we state the central limit theorem for the sample mean of an L^-m- 
approximable functional time series. Its applicability depends on the estimation of the 



5 



covariance kernel of the limit. We therefore also establish the consistency of the kernel es- 
timator of the long-run covariance kernel. We assume that {si} is an L^-m-approximable 
(and hence stationary) functional time series satisfying 

'2 



(2.1) 
and 
(2.2) 



Een = 0, in L' 



J Eel{t)dt < oo. 



Theorem 2.1. //{Op, (Lg), (2J\), (Kg) hold, then 
(2.3) 



N 



N-y^Y.eAz in L\ 



i=l 



where Z is a Gaussian process with 

EZ{t) = and E[Z{t)Z{s)] = c{t, s); 

(2.4) c{t, s) = Eeoit)eois) + ^ Eeoit)e^is) + ^ Eeois)ei{t). 

i>l i>l 

The infinite sums in the definition of the kernel c converge in L'^{[0, 1] x [0, 1]), i.e. c is a 
square integrable function on the unit square. 

Theorem 2.1 is proven in Section [5j 

The kernel c is defined analogously to the long-run variance of a scalar time series. It is 
directly related to the covariance operator of the sample mean defined by 

if \lf 



Cn(x) 



NE 



i=l 



1 ^ 

-Y^E[{ei,x) ej\. 



If the Ei are independent, then Cn{x) = N~'^ J2iLi ^ [(^«' ^) becomes the usual sample 
(empirical) covariance operator, which plays a central role in many exploratory and infer- 
ential tools of functional data analysis of iid functional observations, mostly through the 
empirical functional principal components defined as its eigenf unctions. For functional 
time series, it is not suitable. For any stationary functional time series 

/•/l ,^ 

C7v(x)(t) 

/ \ 



C]\f(t, s)x{s)ds, 
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where 
(2.5) 



CN{t,s)= E (l-^)^ 



[eo{s)ek{t)]. 



The summands in (2.5) converge to those in (2.4), but the estimation of the long-run 



covariance kernel c is far from trivial. 

To enhance the applicability of our result, we state it for the case of a nonzero mean 
function, which is estimated by the sample mean. We thus assume that 



(2.6) 



with the series {si} satisfying the assumptions of Theorem 2.1 



Let be a kernel (weight) function defined on the line and satisfying the following 
conditions: 



(2.7) 



(2i 



(2.9) 



(2.10) 



K(0) = 1, 



K is continuous, 



K is bounded, 



K{u) = 0, if \u\ > c, for some c > 0. 



Condition (2.10) is assumed only to simplify the proofs, a sufficiently fast decay could be 



assumed instead. 

Next we define the empirical (sample) correlation functions 

N 

N 



1 

(2.11) %{t, s) = -Yl (^J- W - W) (^^■-(^) - ^^(^)) ' < ^ < iV - 1, 



j=i+i 



where 



The estimator for c is given by 

(2.12) CM{t, s) = 7o(t, + 5Z ^ ( T ) iliit, s) + 7i(s, t)) 

1=1 ^ ^ 

where h = h{N) is the smoothing bandwidth satisfying 
(2.13) 



N-l 



l<i<N 



h{N) oo, ^^4^^ ^0, as -> oo. 



N 



In addition to (1.2), we also assume that 
(2.14) 



lim m 

m— >oo 



E 



£n{t) - en,m{t)) dt 



1/2 



0. 



Theorem 2.2. Suppose the functional time series {Xi\ follows model (2.6). Under con- 
ditions (Op, (gj, gip, (Q-gj^, gg, 



(2.15) 



2 P 

{cN{t, s) — c(t, s)) dtds ^ 0, 



with c{t,s) by defined by (2.4) and CN{t,s) by (2.12). 



Theorem 2.2| is proven in Section [5j First we use the results of this section in the problem 
of testing the equality of means in two functional samples. 



3 Testing the equality of mean functions 

We consider two samples of curves, Xi, X2, . . . , X^v and X^, X2, . . . X^, satisfying the 
following location models 



(3.1) 



X,{t) = /i(t) + e,{t), XUt) = fi*it) + e*{t). 



The error functions Si are assumed to satisfy the conditions stated in Sections [T] and |2} 
The functions e* are assumed to satisfy exactly the same conditions. In particular, their 
long-run covariance kernel is defined by 

c*{t,s) = Eel{t)el{s) + ^ Ee*(t)e*(,) + Y,Eel{s)e*Xt). 



i>l 



i>l 



We assume that 

(3.2) {si, 1 < z < A^} and {e*, I < j < M] are independent. 
We are interested in testing 

(3.3) Hq : fi = fi* 
against the alternative 

(3.4) Ha: f^^fi*. 



The equality in (3.3) is in the space = L^([0, 1]), i.e. /i = /x* means that J {fi{t) — fi* [t))'^ dt 
0, and the alternative means that / (/i(t) - Ai*(t)) dt > 0. 
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Since the statistical inference is about the mean functions of the observations, our proce- 
dures are based on the sample mean curves 

l<j<Af l<j<J\/ 

The sample means and X\,j- are unbiased estimators of /i and /i*, respectively, so Hq 
will be rejected if 



N,M 



NM 
N + M 



{XM{t)-Xlj{t)Ydt 



is large. 

Before introducing the test procedures, we state two results which describe the asymp- 
totic behavior of the statistic Un^m under Hq and Hj^. They motivate and explain the 
development that follows. 



Theorem 3.1. Suppose Hq, the assumptions of Theorem 2.1 (and analogous assumptions 
for the e* ) and hold. If 



(3.5) 

then 



N 



N + M 



-> 9, for some < 6' < 1, as min(M, A^) -)■ oo. 



T^{t)dt, 



where {T{t), < t < 1} is a mean zero Gaussian process with covariances 
E[T{t)T{s)] = d{t, s) := (1 - e)c{t, s) + ec*{t, s). 



Theorem 3.2. If Ha, and the remaining assumptions of Theorem 3.1 hold, then 



NM 



In particular, if0<6<l, then Un,m oo. 



The kernel d{t,s) in Theorem 3.1 defines a covariance operator D. The eigenvalues of D 
are nonnegative, and are denoted by Ai > A2 > . . .. By the Karhunen-Loeve expansion, 
we have 



(3.6) 



„i 00 

/ T\t)dt = J2^^Nl 
•^0 i=l 



where {Ni, 1 < i < 00} are independent standard normal random variables. 
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Since the eigenvalues Aj are unknown, the right-hand side of (3.6) cannot be used directly 
to simulate the distribution of J T'^{t)dt. We will now explain how to estimate the Aj's. 

Suppose Dn^m is an L^-consistent estimator of D , i.e. 




(3.7) / / diM^Mit, s) - d{t, s) dtds 0, as min(M, A^) ^ oo 



2 p 



We discuss the construction of estimators Dn,m satisfying (3.7) below. For the estimators 



we propose, relation (3.7) holds regardless whether Hq or holds, they do not depend 



on n or /i* either. We will also see that the critical relations (6.1 ) hold under as well as 
under Hq. The distribution of J r'^{t)dt can thus be estimated also under the alternative. 
Let 

Ai = Xi{N, M) > As = X2{N, M)>... 
denote the eigenvalues of Dn,m, i-e. 

(3.8) j ci7v,M(t, s)(^i(s)ds = Ai(^i(t), 

where the 0i{t) = 0i{t] N, M) are the corresponding eigenfunctions satisfying j 0f{t)dt = 
1. Choosing p so large that X]r=i ^ large percentage of Yli=i^^ -^i' "^^^ approximate 
the distribution of jT^it)dt by that of XlLi \N^- 

The statistical inference is based on the difference — ^li- Observe that 

^^^E [{X^it) - X^t)) {Xj,is) - X^s))] ^ d{t, s), as min(M, N) ^ oo, 

that is, d is the asymptotic covariance kernel of the difference Xn — X^.j. We therefore use 
projections onto the eigenfunctions (fi,(p2,---,fp associated with the p largest eigenvalues 
of D. This is analogous to projecting onto the functional principal components in one 
sample problems, as these form an L^-optimal orthonormal basis. Without any loss of 
generality, we assume that the (pi,ip2, . . . ,(pp form an orthonormal system (the ipi are 



orthogonal under (3.13), so only a normalization to unit norm is required). We define the 
projections 

(3.9) tti = {Xn - Xli, ipi) , 1 <i <p, 

and the vectors 

a = [ai,a2, . . . ,apf. 
We show in the proof of Theorem 3.3| that 



1/2 

(3.10) ^WV]^) ^^N,(0,Q), 
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where Np(0, Q) stands for the p-variate normal random vector with mean zero and the 
covariance matrix Q = diag(Ai, A2, . . . , Ap). Since the operator D is unknown, we can- 
not compute the ipi. However, any estimator for D satisfying (3.7) can be used to find 
estimates for the tpi. Let (^j be the empirical eigenfunctions defined by (3.8), and set 

hi = (Xn - Xl^, <^i) , l<i<p. 



The limit relation (3.10) suggests the following statistics: 

MN 



(3.11) 

and 
(3.12) 



U 



(1) 

N,M 



M + N 



U 



(2) 



MN 



1=1 



The following theorem establishes the limits of ul^\.j and f/^^j^f under Hq. 



Theorem 3.3. Suppose Hq, the remaining assumptions of Theorem 3.1, (3.7) and 



(3.13) 

hold. Then 
(3.14) 



Ai > A2 > . . . > Ap > Ap+i 



1=1 



Np are independent standard normal random variables, and 



where Ni,N2, 
(3.15) 

where x^ip) stands for a chi-square random variable with p degrees of freedom. 

We note that U^\.j is essentially the first p terms in the Karhunen-Loeve expansion of the 
integral in the definition of Un.m- Thus, the limit in (3.14) is exactly the random variable 
we used to approximate the distribution of Un,m- The limit in (3.15) is distribution free. 
The consistency of the tests based on Ul^\j and U^\,j follows from the following result. 



Theorem 3.4. Suppose Ha, the remaining assumptions of Theorem 3.1, (3.7) and (3.13) 
hold. Then 

N±M_ , 2 



i=l 
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and 



iV + M (2) P 



i=l 



In particular, if < 9 < 1 in (3.5), then 
— fi*, 7^ for at least one 1 < i < p. 



X,. 



A oo and U^\,j A oo, provided 



We see that the condition for the consistency is that ^ — ^* is not orthogonal to the hnear 
subspace of spanned by the eigenfunctions 99 j, 1 <i <p. 



The implementation of the tests based on Theorems 3.3 and 3.4 depends on the existence 
of an estimator of the kernel d{t, s) which satisfies (3.7). The remainder of this section is 
dedicated to this issue. 

The estimation of D is very simple if the Ei are iid, and the e* are iid. In this case, setting, 

N 
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N + M' 



we can use 

(3.16) 

where 



f^7V,M(^, s) = (1 - 9)cN{t, s) + 9clj{t, s), 



1 ^ 



i=l 
M 



By condition (2.2), we can use the weak law of large numbers in a Hilbert space to 
establish (3.7). The estimation of D is much more difficult if only (1.2) is assumed, and 
its asymptotic justification relies on Theorem 2.2[ Recall the definition of the estimator 
CAr(t, s) given in (2.12), and define the estimator Cj^(t, s) fully analogously. Our estimator 
for d(t,s) is then (3.16) with C]\f(t,s) and c*p^{t,s) so defined. The following result then 
follows directly from Theorem |2.2[ 



Theorem 3.5. Suppose the functional time series {Xj} satisfies the assumptions of The- 
orem \2.2, and the series {X*} satisfies the same assumptions stated in terms of the e*. 



If (3.2) holds, then (3.7) holds. 



We emphasize that under the conditions of Theorem 3.5 relation (3.7) holds both under 
Hq and Ha- 

We now focus on the numerical issues related to the computation of the Sj and the Aj 
appearing in the definitions of statistics U^\j and U^^\j. The and the Aj require the 
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computation of the eigenfunctions and the eigenvalues of the operator D. Except in the 
case of independent observations in each of the two samples, these quantities cannot be 
computed using existing software because ID is not an empirical covariance operator of a 
functional iid sample. We recommend the following algorithm which we used to implement 
the tests. Let {e^,£ > 1} be an orthonormal basis. The results reported in Section |4] are 
based on an implementation which uses the Fourier basis. In order to find approximate 
solutions to 



(3.17) j dNMt, s)(l)is)ds = A0(t), 

we use approximate expansions for (f){s) and dN,M{t, s): 

49 

k=l 
49 49 

k=l 1=1 

The coefficients 0^ and d^/ are given by 



(t)k= I <p{t)ek{t)dt 

and 

(3.18) dk/= I I dN,M{t,s)ek{t)ee{s)dtds. 




We replace 0(s) and d]\f^M{t,s) in the left side of (3.17) with the above expansions to 
obtain 



49 49 



(3.19) ^^4,£efc(t)^A0(t). 



fc=i e=i 



Multiplying both sides of (3.19) by ej and integrating yields 



49 

1=1 

In matrix form this is 

D0 = A0, 

where D = [dj^i, 1 < J, ^ < 49] and <p = [01, {/)2, . . . , (/)49]'^. Thus we have reduced 



the problem of finding solutions of (3.17) to finding eigenvalues and eigenvectors of the 
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matrix D. Let [(f)m,i, <Pm,2, ■ ■ ■ , 4>m,49]'^ be the eigenvector corresponding to the mth largest 
eigenvalue of D. Then the eigenfunction associated with the mth largest eigenvalue of 

[Xn - XIj, (pi) 



dN,M{t, s) is approximately Y2T 4'm,eGe{t)- Using this notation, we obtain 



(3.20) 



49 



We then obtain U^\j and U^'j^^j as per (3.11) and (3.12) 



(2) 



N M 



To complete the description of the test procedures, we must specify how the value of p in 



the definitions of ?7^^^ and U^'^^ is selected. This issue has been extensively studied in 
one sample problems, and several approaches have been put forward, including cross val- 
idation and penalty criteria. In our experience (for smooth densely recorded curves), the 



r(2) 



simple cumulative variance method advocated by Ramsay and Silverman (2005) has been 
satisfactory. We therefore recommend to use p such that the first p empirical functional 
principal components in each sample explain about 85% of the variance. As we will see 
in the data examples studied in Section |4| it is typically useful to look at the P-values 
for a range of p's. 



4 A simulation study and data examples 

We begin by presenting the results of a simulation study intended to evaluate the empirical 
size and power of the testing procedures introduced in Section |3j We then illustrate their 
properties on two data examples. 

4.1 A simulation study 

In this section, we compare the the performance of the tests based on statistics f/^^^,/ and 

(2) 

M using simulated Gaussian functional data. We consider all combinations of sample 
sizes N,M = 50, 100, 100, and each pair of data generated processes was replicated three 
thousand times. To investigate the empirical size, without loss of generality, we set 
fi{t) = fi*{t) = 0. Under the alternative, we set fi{t) = and fi*{t) = at{l — t). The power 
is then a function of the parameter a. We considered two settings for the errors: 

1. Both the ei{t) and the B*{t) are iid Brownian bridges. 

2. Both the ei{t) and the B*{t) are functional AR(1) (FAR(l)) processes with the kernel, 

= —I — . 

4 Jg ax 
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That is, the error terms, ei{t), follow the model 



where Bi{t) are iid Brownian bridges. 

We calculated the test statistics f/^\^ and U^\^ as explained in Section 3j These statistics 
depend on the choice of the weight functions K and K*, and the bandwidth functions h 
and h*. A great deal of attention has been devoted over several decades to the optimal 
selection of these functions for scalar and vector-valued time series, and we cannot address 
this issue within the space of this paper. We follow the recommendation of ? and use, 
for both samples, the flat top kernel 



1.1 - \t\ 





< |t| < 0.1, 
0.1 < \t\ < 1.1, 

1.1 < \t\ 



with h = N^^^ and h* = M^/^. We emphasize that this full estimation procedure was 
used for all data generating processes, including those with independent errors. 



Table 4.1. Power of Test (in %) using [/, 



(1) 

100,200 



and U-, 



(2) 

100,200 



with iid Brownian-bridge errors 





a = .01 


a = .05 


a = .10 


a 


•^100,200 


'-^100,200 


•^100,200 


•^100,200 


•^100,200 


•^100,200 


0.0 


1.5 


1.5 


6.3 


6.2 


11.4 


11.6 


0.1 


2.5 


3 


7.4 


8.0 


13.2 


13.6 


0.2 


6.0 


4.4 


16.7 


13.0 


24.8 


20.2 


0.3 


14.2 


9.2 


30.4 


23.2 


41.3 


33.0 


0.4 


26.4 


17.0 


48.5 


36.1 


60.8 


48.0 


0.5 


44.0 


31.4 


64.7 


53.5 


75.2 


64.3 


0.6 


59.4 


45.9 


80.3 


68.0 


87.9 


78.4 


0.7 


78.0 


64.7 


91.8 


82.4 


96.0 


89.0 


0.8 


88.0 


78.0 


95.9 


90.9 


98.0 


94.6 


0.9 


94.2 


88.1 


98.7 


95.8 


99.4 


97.9 


1.0 


98.0 


94.8 


99.5 


98.5 


99.9 


99.3 


1.1 


99.6 


98.4 


100.0 


99.8 


100.0 


100.0 


1.2 


99.9 


99.4 


100.0 


99.9 


100.0 


100.0 


1.3 


100.0 


99.8 


100.0 


100.0 


100.0 


100.0 
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Table 4.2. Power of Test (in %) using f^iog 200 ^100200 "^it^ FAR(l) errors 





a = .01 


a = .05 


a = .10 


a 


"^100,200 


'-^100,200 


'^100,200 


'^100,200 


•-^100,200 


'^100,200 


0.0 


1.8 


1.9 


6.6 


7.2 


12.2 


13.5 


0.1 


2.4 


2.2 


7.9 


7.7 


13.5 


14.5 


0.2 


5.1 


3.3 


13.6 


11.6 


21.6 


18.7 


0.3 


9.8 


6.3 


23.6 


17.6 


34.5 


26.8 


0.4 


19.4 


12.3 


35.9 


26.5 


46.7 


36.3 


0.5 


26.8 


19.5 


47.9 


38.6 


60.4 


49.7 


0.6 


42.1 


29.6 


62.2 


51.8 


73.1 


62.5 


0.7 


56.4 


42.8 


75.4 


63.8 


83.2 


74.0 


0.8 


68.6 


53.8 


85.7 


74.6 


91.5 


83.1 


0.9 


80.8 


67.6 


92.7 


85.9 


96.4 


91.9 


1.0 


87.4 


78.7 


95.9 


90.8 


98.1 


94.5 


1.1 


93.7 


86.8 


97.9 


95.8 


99.1 


97.6 


1.2 


97.6 


93.7 


99.5 


98.1 


99.8 


99.2 


1.3 


98.5 


96.4 


99.7 


98.9 


99.9 


99.6 



The results of the simulation study can be summarized as follows. The empirical size of 
the tests is larger in the case of FAR(l) errors. When a increases to 0.2 or larger, the 
empirical power of the test is smaller in the case of FAR(l) errors. Thus increasing the 
dependence in the error terms increases the size and decreases the power of the test. In 
both cases the tests have have a slightly larger-than-nominal size and very good power. 
These observations are illustrated in Tables 14.11 and 14.21 Based on the whole simulation 
study, we can conclude that the performance of both tests is better if the sample sizes 
and M are about equal. For example, for N = M = 100, the empirical sizes are closer 



to the nominal sizes then in the case = 100, M = 200 shown in Tables 4.1 and 4.2 



The power is very high even for small sample sizes. This is illustrated in Figure 4.1 which 
shows the samples with A^ = M = 50 and with slightly different means (a = 0.8). Visual 
inspection does not readily lead to the conclusion that the samples in the left and right 
panels of Figure 4.1| have different means, yet our tests can detect it with a very high 
probability. None of the two test statistics clearly dominates the other for the simulated 
Gaussian data, but a difference in behavior can be see when the tests are applied to real 
data sets, to which we now turn. 
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Figure 4.1. Fifty trajectories of the Brownian bridge (left) and fifty independent trajectories 
of the Brownian bridge plus = 0.8t(l — t) (right). The tests can detect the different means 

with probability close to 90%. 




—\ \ \ 1 \ \ — — I \ \ 1 \ r~ 

0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 



4.2 Mediterranean fruit flies 

In our first example, it can be assumed that the curves in each sample are independent, as 
they were obtained from a randomized experiment. The data set used in this example was 
kindly made available to us by Hans-Georg Miiller. It was extensively studied in biological 



and statistical literature, see Miiller and Stadtmiiller (2005) and references therein. We 



consider 534 egg-laying curves (count of eggs per unit time interval) of medfiies who lived 
at least 30 days. Each function is defined over an interval [0,30], and its value on day 
t < 30 is the count of eggs laid by fly i on that day. The 534 flies are classified into 
long-lived, i.e. those who lived longer than 44 days, and short-lived, i.e. those who died 
before the end of the 44th day after birth. In the data set, there are 256 short-lived, 
and 278 long-lived flies. This classification naturally defines two samples: Sample 1: the 
egg-laying curves {Xj(t), < t < 30, 2 = 1,2,..., 256} of the short-lived flies. Sample 2: 
the egg-laying curves {X*(t), < t < 30, j = 1, 2, . . . , 278} of the long-lived flies. The 



egg-laying curves are very irregular; Figure 4.2 shows ten smoothed curves of short- and 



long-lived flies. The tests are applied to such smooth trajectories. 



Table 4.3 shows the P-values as a function of p. For both samples, p = 2 explains slightly 
over 85% of the variance, so this is the value we would recommend using. Both tests 
reject the equality of the mean functions, even though the sample means, shown in Figure 



4.3 are not far apart. The P-values for the statistic f/^^^ are much more stable, equal 
to about 1%, no matter the value of p. The behavior of the test based on f/^^^ is more 
erratic. This indicates that while the test based on [/'-^^ is easier to apply because it uses 
standard chi-square critical values, the test based on U^^^ may be more reliable. 
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Figure 4.2. Ten randomly selected smoothed egg-laying curves of short-lived medflies (left 
panel), and ten such curves for long-lived medfies (right panel). 




Table 4.3. P-values (in percent) of the tests based on statistics Uj^ j^^^ and Uj^ applied to 
medfly data. 



p 


1 


2 


3 


4 


5 


6 


7 


8 


9 




1.0 


1.0 


1.0 


1.1 


1.1 


1.0 


1.0 


1.1 


1.1 


^7(2) 


1.0 


2.2 


3.0 


5.7 


10.3 


15.3 


3.2 


2.7 


5.0 
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Figure 4.3. Estimated mean functions for the medfly data: short hved -sohd hne; long hved 
-dashed hne. 




4.3 Eurodollar futures contracts 



Our next example uses financial data kindly made available by Vladislav Kargin. This 



data is used as an example of modeling with the functional AR(1) process in Kargin 



and Onatski (2008) The curves, one curve per day, are constructed from the prices of 



Eurodollar futures contracts with decreasing expiration dates. The seller of a Eurodollar 
futures contract takes on an obligation to deliver a deposit of one million US dollars to 
a bank account outside the United States i months from today. The price the buyer is 
willing to pay for this contract depends on the prevailing interest rate. These contracts 
are traded on the Chicago Mercantile Exchange and provide a way to lock in an interest 
rate. Eurodollar futures are a liquid asset and are responsive to the Federal Reserve 
policy, inflation, and economic indicators. 

The data we study consist of 114 points per day; point i corresponds to the price of a 
contract with closing date i months from today. We consider 4 samples, each consisting 
of 100 days of this data: 

Sample 1: curves from September 7, 1999 to January 27, 2000. 
Sample 2: curves from January 24, 1997 to June 17, 1997. 
Sample 3: curves from December 4, 1995 to April 24, 1996. 
Sample 4: curves from March 6, 2001 to July 26, 2001. 



Figure [44] shows the sample mean functions for the four samples. If a significance test does 
not reject Hq, we can conclude that the expectations of the future evolution of interest 
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rates are the same for the two periods over which the samples were taken. A rejection 
means that these expectations are significantly different. As the analysis below reveals, 
we can conclude that expectations of future interest rates were different in Spring 1996 
than in Summer 2001. 

Figure 4.4. Sample means of the Em'odollar cmves: Left: sample 1 solid; sample 2 dashed. 
Right: sample 3 solid; sample 4 dashed. 




Table 4.4 shows the P-values as a function of p when the test is applied to samples 1 and 
2, and also when it is applied to samples 3 and 4. In both samples 1 and 2, p = 1 explains 
more than 94% of the variance, in both samples 3 and 4, p = 1 explains more than 84% 
of the variance. Thus, following the recommendation of Section |3| we use the P-values 
obtained with p = 1. They lead to the acceptance of the null hypothesis of the equality 
of mean functions for periods corresponding to samples 1 and 2, and to its rejection for 
periods corresponding to samples 3 and 4 (notice that the 0.81 in the bottom panel of 
Table 4.4 is 0.81%). These conclusions agree with a visual evaluation of the sample mean 
functions in Figure 4A They also confirm the observation made in Section 4J^ that the 
tests have very good power, as the curves in the right panel of Figure 4.4 are not far 



apart. Both graphs in Figure [4^ give us an idea what kind of differences in the sample 
mean functions are statistically significant, and which are not. 



4.4 Conclusions 



The simulations and data examples of this section show the tests we propose enjoy good 
finite sample properties. Tests of this type allow us to quantify statistical significance 
of conjectures made on the basis of exploratory analysis. For example, the sample mean 
curves in Figure 4.3 and the right panel of Figure 4.4| look a bit different, but a significance 
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Table 4.4. P-values (in percent) of the statistics applied to Eurodollar data; samples 1 and 2 
(top), samples 3 and 4 (bottom). 



p 


1 2 3 4 5 




38.49 39.17 37.12 37.15 35.50 
38.49 68.52 0.00 0.00 0.00 



P 


1 2 3 4 5 


f/(i) 

f/(2) 


0.81 0.23 0.10 0.01 0.07 
0.81 0.00 0.00 0.00 0.00 



test allows us to state with confidence that they correspond to different population mean 
functions. 

In many procedures of functional data analysis, both exploratory and inferential, the issue 
of choosing an optimal dimension reduction parameter, like the p in our setting, is delicate. 
Therefore, procedures less sensitive to such a choice are preferable. From this angle, the 
Monte Carlo test based f/^^^ is preferable, as an inspection of Tables 4^ and AA_ reveals. 
The test based on f/^^^^ is however easier to apply, and in our examples and simulations 
leads to the same conclusions if p is chosen according to the cumulative variance rule. 

The data examples of this section also show that the optimal value of p is typically a small 
single digit number, 1 or 2 in our examples. Therefore, developing asymptotics as p tends 
to infinity is not necessary, and may, in fact, be misleading because for larger values of p 
the tests may yield counterintuitive results. 



5 Proofs of the results of Section 2 



2.1 



The proof is done in two steps. First we show that X]i=i ^«(^) 



Proof of Theorem 
is close to £j,m(^), if ^ is sufficiently large. Then we establish (2.3) for 

dependent functions for any m > 1. 

As the first step, we show that 



m- 



(5.1^ 



lim sup lim sup E 

m— >oo Af— )-oo 



N 



i=l 



dt = 0, 
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where the variables ei^m are defined in (1.3). By stationarity, 

2 



E 



^ {ei{t) -Ei^rnit)) 



.l<i<N 



l<i<N l<j<N 

= NE{eoit)-6o,^{t)f + 2 J2 E{e^{t)-ei,m{t)){ej{t)-ej,^{t)). 

l<i<j<N 

In the proof, we will repeatedly use independence relations which follow from represen- 



tations (1.1) and (1.3). First observe that if j > i, then is independent of Sjj-i 

because 



Consequently, E {ei{t) — Ei^mit)) ejj-i(t) = 0, and so 

l<i<j<N l<i<j<N 



Using the Cauchy-Schwarz inequality and (1.2), we conclude 



l<i<j<N 



■J ^ ^ X ^ A.^ AT 



l<i<j<N ' 
l<i<j<N '- 



E {e^{t) - ei,^{t)f dt 



1/2 



r /■ \ f 

E {eo{t) - eo,n{t)f dt E 

l<i<j<N L"' -I L"' 

/ E (£o,™(t) - eo{t)f dt J2 (^o{t) - SoAt)) 

-I k>l 



{eo{t) - eoj-i{t) fdt 



1/2 



1/2 



Hence 



lim sup lim sup — 



.1/2 



dt 



I Y E[{ei,rr,{t)-ei{t))e,{t)]dt 



\<i<3<N 



0. 



Similar arguments give 



lim sup lim sup — 

N 



J2 E[i6i,rnit)-eiit))ej,mit)]dt 
l<i<j<N 
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Completing the verification of (5.1). 

Tlie next tlie step is to sliow tliat N^^^"^ X]i<i<Ar^i,m converges to a Gaussian process Zm 
witli covariances defined analogously to (2.4). Recall that for every integer m > 1, {Si^m} 
is an m-dependent sequence of functions. To lighten the notation, in the remainder of 
the proof, we fix m and denote sequence {ej,m} by so {ei} is now m-dependent. 
Let i^' > 1 be an integer and ipi be an orthonormal basis determined by the eigenfunctions 
of Ee{t)e{s). The corresponding eigenvalues are denoted by Ui. Then, by the Karhunen- 
Loeve expansion, we have 



i>i 



Next we define 



1/2 



i<e<K 

By the triangle inequality we have that 

_l<i<N 



dt 



1/2 



dt 



+ <^ E 



jey(m-i) 



+ . . . 

2 ^ 1/2 

dt 



where V{k) = {i '■ 1 < i < N,i = k (mod m)},0 < k < m — 1. Due to the m depen- 
dence of the sequence {Si}, X]iey(A;)(^*(^) ~ ^ independent, identically 
distributed random variables, and thus we get that 



E 



-\ 2 



iey(m-l) 



dt<Nj2E{Xo,iJ, 



e>K 



Utilizing 



hm E^(^o,^^)' = 



i>K 



we conclude that for any x > 



dt > X 



0. 
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The sum of the e^^^'s can be written as 



ATI/: 



Next, we use the central hmit theorem for stationary m-dependent sequences of random 
vectors (see ? and the Cramer- Wold theorems in ?, pages 9 and 120)) and and get that 

I l<i<N J 

where Ni^(0, A/^) is a i^'-dimensional normal random variable with zero mean and co- 
variance matrix = diag(i/i, . . . , u^)- Thus we proved that for all i^' > 1 

l<i<N l<e<K 

where Ni,i > 1 are independent standard normal random variables. It is easy to see that 
/ f E ^y^NeMt)] dt= J2 ^^^i ^ 0' 

\K<e<oo J K<l<oo 

as A' — )■ oo. Thus we have the convergence of ^"^''^ X]i<i<Af therefore 
the proof of the Theorem is now complete. 



Proof of Theorem 2.2, First we reduce (2.15) to (5.2). Then, we reduce (5.2) to 



(5.6). 



Since 



1 ^ 

i=l 



we obtain that 



{%it,s) - E[eoit)eois)]f dtds 
+ A(^j (X^it) - fi{t)Y dt 



— /u(s)) — -E[£:o(i)£^o('5)] 1* dtds 



Op{l), 
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using the ergodic theorem for random variables in a Hilbert space. 
Next observe that 



1 ^ 



+ 



j=i+i 
N-i 



N 



\ j=i+l / \ j=i+l ) 



Therefore, setting, 



we obtain 



Af-l 



1 ^ 



1=1 



iV-l 



1=1 



i=i ^ ^ I i=i+i ) 
j=i ^ I j=i+i ) 



i\ N -i 



h N 



€N{t)eN{s). 



gud 

By stationarity we conclude that for any 1 < i < N , 

2 

/■ / 1 ^ \ 



Eel{t)dt + ^f2 J Eeo{t)ej{t)dt . 
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Since Eq and Ejj are independent, we get by (1.2) 



E 



Eeo{t)ej{t)dt 



oo 



\ 1/2 



1/2 oo 



<(eJ sl{t)d?j ' ^ (^J E{e,{t)-e,,{t)fdt^ 



y/2" 

1/2 



< OO. 



Thus, we have 



max E 

l<i<N 



Consequently, using the triangle inequality. 



N-i / • \ / /. r 1 ^ 

j=i ^ ^ \ L i=j+i 



1/2 



ejsf{s)ds 



1/2 



iV 



0(1) = 0(1), 



on account of (2.13). 



Hence to establish (2.15) , it is enough to prove that 



(5.2) 
where 



i=l 



i-j 7i(t, s) - ci{t, s) dtds = op(l). 



i>l 



Let {en,m, —oo < n < oo} be the random variables defined in (1.3), where m is a fixed 
number. Let 



7i. 



1 ^ 

i,m(^; -S) ^ ^ ^ ^j,mit)^j—i,m{^)- 



j=i+l 
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We show that for every m > 1 



(5.3) 
where 



i=l 



We also note that (1.3|) and (|l.2|) imply 
(5.4) 



lim / / (c^r\t, s) - ci{t, s)) dtds = 0. 

m—>-co J J \ / 

Since {en,m, —oo < n < oo} is an m-dependent sequence, 

m 



i=l 



Using (2.7), (2.8) and (2.13), we get 



max 

l<i<m 



0, as cx). 



By the ergodic theorem. 



{li,m(t, s) - E[ei.m{s)ei+i^rn(t)]f dtds = Op{l) 



for any fixed i. Hence (5.3) is proven, once we have shown that 
(5.5) 1 1 { J2 ^fll7i,m(t,s) ] dtds = op{l) 



\i=m+l 



It is easy to see that 

N-l 



E 



\i=ra+l 



^ \ t] 1i,m{t, s) dtds 



h h N-l N-l 



^ E EE Ex 



i=m+l e=m+l k=i+l n=i+l ^ ^ 

provided h < N — 1. The sequence {en^m, — c>o<'ri<oo}isan m-dependent, and there- 
fore Ek^m and Ek-i.m are independent, since i > m + 1. Similarly, en,m and en-e,m are 
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independent. Hence the number of terms when E [ek,m£k-i,m£n,m£n-e,Tn] is different from 
zero is 0{Nh). Consequently, 



E 



N-l 



J2 T 7^,m(t,s) dtds = 



0(1). 



This completes the verification of( 5.5) 
Next we show that for all e > 0, 



(5.6) lim limsupP 



[7j(t, s) - 'Ji,m{t, s)] dtds > e > = 0. 



Using the definitions of the covariances 7j(t, s) and Ji^mit, s), we consider the decomposi- 
tions 



i=l ^ ^ j=i+l 

m h 



^j,in ('^)^j— i.m ("S)] 



i=\ i=m+l J ^ ' j=i+l 



N 



and 



Clearly, 



^E^(^) E {eAt)-HAt))eJ-^{s)) dtds 

i=l ^ ^ j=i+l 



1/2 



1 

iV 



i=l 



1/2 

(e,(t)-£,-„(t))'rft|^ <j / e^^_,{s)ds 



1/2 



so, by (2.14) 



E 



1 / « 



1/2 



i=l ^ ^ i=*+i 



<mi^E j {eo{t) - eo,m(t)f dtE j el{s)ds 
< Am \E {£o(t) — eo^rnit)) dt > — )• 0, as m — )■ oo. 
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according to (2.14), where A is a constant. 



Next we use the decomposition 
to get 



i=m+l 

oo N 



< 



V V 

s) j dtds > 

i=m+l j=i+l J 



Therefore, by (2.2) and (1.2), we have 



E 



1 



i=m+l 
oo N 



N 



1/2 



j=i+l 



< 



< 



i=m+l j=i+l 
oo 



i^jit) - ^jAt)? dt 



{ej{t) - ej^i{t)f dt 



^^^^ 

i=m+l j=i+ 
^ oo N 

iv E E 

(Eoit) - eo,iit)ydt 
We have shown so far that for any e > 0, 



1/2 



- 1/2 









e^_,{s)ds 

1/2 



1/2- 



i=m+l 



1/2 



— )• 0, as m — )■ oo. 



hm hmsupP 

m-S>oo jv^>oo I 



^ \ i=m+l 



N 



^(h) Yl ^At)ej-^{s) ] dtds>e\=0. 

j=i+i 



Similar arguments give 
hm hmsupP 



m^oo N^oo 



I \ i=m+l 



N 



E ^ (h) E ^j,m{t)£j-i,m{s) ) dtds > e J> = 0. 



j=i+l 



This completes the verification of (5.6), so (5.2) is proven 
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6 Proofs of the results of Section 



In the proofs, we will often use the relations 



(6.1) max |Aj — Aj| — )■ 0, and max — Cj(^j|| — as min(M, A^) — )■ oo, 

where q = sign ((<^j, , Analogous relations have been extensively used for the eigen- 
values and the eigenfunctions of the empirical and population covariance operators, see 



Bosq (2000) Gabrys and Kokoszka (2007) , Horvath et al. (2010) ,|Panaretos et al. (2010) 



among many others, but they hold in much greater generality, see Chapter 2 of [Horvdth 
and Kokoszka (2011) Under (3.13), they hold for the eigenelements of the operators I) 
and D defined in Section [HI 



Proof of Theorem 13.11 By Theorem [23| assumptions (|LT|-(pr2|) imply that, as N ^ 
oo and M ^ oo 

\ l<i<N i<j<N ) 

where F*^^) and V^'^^ are independent L^-valued mean zero Gaussian processes with covari- 
ances 

E [r(i)(t)r«(5)] = c(t,3), E [r(2)(t)r(2)(,)] = c\t,s). 



Hence 
(6.2) 



NM 

N + M 



[ i^'' dtAf r{t)dt 

\ l<i<N l<j<M J 



where r(t) = (\-efl'^V^^\t)^e^l'^V^'^^{t).ll\\.&co\\.Q\\\&\o\YoiT\\.'^ox&\^?i\ □ 



Proof of Theorem 13.21 By the ergodic theorem in a Hilbert space, | |Xjv | = op(l) 
and \ \Xm ~ /^*|| = Op(l), which imply the result. □ 



Proof of Theorem 13.31 Under assumptions (2.2) and (1.2), we have, jointly, 



and 



I l<i<Af J 

l<J<Af J 
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where 'N^^ (O, Q*-^^) and Np^'' (O, Q*-^-*) are independent p-dimensional normal random 
vectors. Since 

l<i<Af l<i<M 



( pUOl holds with Q = (1 - ^)Q(^) + 0Q(2). 

Next we observe that the matrix Q = {Q{k,£), I < k,£ < p) satisfies 



Q{k,e) = J J 



d{t, s)ipk{t)(p£{s)dtds = XkSke, 



where Sij is Kronecker's delta, using the fact that the ipi are orthonormal eigenfunctions 
and the A, are the corresponding eigenvalues. 



Recall now relations (6.1), and observe that 



Ofc — Cfc 



\ l<i<N l<j<M J 

+ (iV- e, - M-' J2 

\ l<i<N l<j<l 



£i , Vfc - Ck^k 



By (6.1) and (6.2), 



NM 
N + M 



1/2 



l<i<N 



< Ull^i ( / {0k - Ckipkfdt 

op(l 



i<i<A'/ 



1/2 



Hence the result follows immediately from (3.10) and the diagonality of Q 



□ 



Proof of Theorem 13.41 By the ergodic theorem ai {fi - fj,* ,ipi) , 1 < i < p. Since 



relations (6.1 ) hold also under Ha, the result is proven. □ 
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